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Abstract 

A general primal-dual splitting algorithm for solving systems of structured coupled monotone 
inclusions in Hilbert spaces is introduced and its asymptotic behavior is analyzed. Each inclusion 
in the primal system features compositions with linear operators, parallel sums, and Lipschitzian 
operators. All the operators involved in this structured model are used separately in the proposed 
algorithm, most steps of which can be executed in parallel. This provides a flexible solution 
method applicable to a variety of problems beyond the reach of the state-of-the-art. Several 
applications are discussed to illustrate this point. 
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1 Introduction 

Traditional monotone operator splitting techniques fl8l[]3|23l|2S|28llMIMIMES^ have their 
roots in matrix decomposition methods in numerical analysis II2TI I44H and in nonlinear methods for 
solving optimization and variational inequality problems [71 (TTJ |30l [33l |39l . These methods are 
designed to solve inclusions of the type G B\x + B2X, where B\ and B2 are maximally monotone 
operators acting on a Hilbert space %. Extensions to sums of the type G Ylk=i ^kx are typically 
handled via reformulations in product spaces [8l I40H . In recent years, new splitting algorithms have 
emerged for problems involving more complex models featuring compositions with linear opera- 
tors [13] and parallel sums K19[ l45j] (see Q1.9D "). These algorithms rely on reformulations of the 
inclusions as two-operator problems in a primal-dual space, in which the splitting is performed via 
an existing method. This construct makes it possible to activate separately each of the operators 
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present in the model, and it leads to flexible algorithms implementable on parallel architectures. 
In the present paper, we pursue this strategy towards more sophisticated models involving systems 
of structured coupled inclusions in duality. The primal-dual problem under consideration is the 
following. 

Problem 1.1 Let m and K be strictly positive integers, let (%j)i^ m and (Qkji^k^K be real Hilbert 
spaces, let (jUj)i<i< m G [0, +oo[ m , and let {vk)i^K G [0, +oo[ x . For every i G {1, . . . , m} and k G 
{1, . . . , K}, let Q : Hi — )■ Hi be monotone and ^j-Lipschitzian, let A^. Hi — > 2 Wi and B k : Q k ^ 2 Qk 
be maximally monotone, let D k : Q k ^ 2 Sk be maximally monotone and such that D^ 1 ■. Q k ^ Q k is 
ffc-Lipschitzian, let Zi G "Hi, let G an d let -^fcj G £ It is assumed that 



B = max] max iii, max z/z, > + a/X > 0, where A G 

llsCisCm l^k^K > 

and that the system of coupled inclusions 
find xx G Hi, . . . ,x^T G H m such that 

( K 



K 



Em II 1 1 2 
i=ilNII 2 <iit=i i=i 



+oo 



(l.D 



XV / / '/ft 

fc=i ^ ^ i=i 

G vl m x^ + ^ L* km ( (B k DD k )(^2 L ki%i ~ r k J J 
k=i ^ M=l ' ' 



LkiXi - r k + Cixi 



+ C m x r 



(1.2) 



possesses at least one solution. Solve (I1.2D together with the dual problem 



find G <?i, 



. G Qk such that 



/ft / 1\ \ 

ri G -^Lii(Ai + C i )~ 1 ( Zi -J2 L kiVk) +B^ l Ul + D^vl 
i=l ^ fe=l ' 

-r x G - Y L Ki (Ai + Ci) _1 ( Zi - Y L tiVk) + By/W + D^W- 

i=l ^ k=l ' 



(1.3) 



The primal system 01 .2D captures a broad class of problems in nonlinear analysis in which m vari- 
ables x\, . . . , x m interact. The ith inclusion in (II .2D features two operators Ai and Cj which model 
some abstract utility of the variable Xi, while the operator (B k )i^ k ^K> {D k )i<^ k <^K: an d (-kfci)i<i<m 

l^k^K 

model the interaction between x% and the remaining variables. One of the simplest realizations of 
(II .2D is the problem considered in IfTOll , namely 



find x\ G %, x 2 G % such that 



G A\x\ + x\ — X2 

G A 2 X2~ -X~[ + X2~, 



(1.4) 



where (H, \\ ■ ||) is a real Hilbert space, and where A\ and A 2 are maximally monotone operators 
acting on %. In particular, if A\ = dfi and A 2 = df 2 are the subdifferentials of proper lower 
semicontinuous convex functions fi and f 2 from % to ]— oo, +oo], (I1.4D becomes 



minimize + /2(x 2 ) + ~||xi - x 2 \\ 2 . 



(1.5) 
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This formulation arises in areas such as optimization [ 1 ] , the cognitive sciences [5] , image recov- 
ery Il20ll . signal synthesis 11291 , best approximation (91], and mechanics H37H . In Il3]], we consid- 
ered the extension of (11.51) which amounts to setting in Problem ll.li for every i G { 1 , . . . , m} and 
k G {1, . . . , K}, = Cj = 0, and = Vg^, where fa : % — > ]— 00, +00] is a proper lower 
semicontinuous convex function and ; Gk — > K is convex and differentiable with a Lipschitzian 
gradient. This leads to the minimization problem 



which has numerous applications in signal processing, machine learning, image recovery, partial 
differential equations, and game theory; see Q2J [6l [12], [141 |25l [27l HTll and the references therein. 
In the case when m = 1 in Problem ll.li and under certain restrictions on the operators involved, 
primal-dual algorithms have been proposed recently in |[T3l [T9l 1451 . On the other hand, a primal 
algorithm was proposed in |[3]] for solving systems of inclusions of type (11.2ft in which the operators 
(Ci)i^i^m and (D7 )i^k^K are zero, and the coupling operators (-Ba^ka^a - are restricted to be 
single-valued and to satisfy jointly a cocoercivity property. 

The goal of the present paper is to develop a flexible algorithm to solve Problem 11.11 without 
the restrictions imposed by current methods. In particular, no additional hypotheses will be placed 
neither on the coupling operators (B^i^k^K an d (Dk)i^k^K, nor on the number m of variables. 
In the proposed parallel splitting algorithm, the structure of the problem is fully exploited to the 
extent that the operators are all used individually, either explicitly if they are single-valued, or by 
means of their resolvent if they are set-valued. The main algorithm is introduced and analyzed in 
Section [2] The remaining sections are devoted to applications to problems which are not explicitly 
solvable via existing techniques. Thus, in Section [3j we discuss applications to univariate inclusion 
problems featuring general parallel sums, in the sense that the operators {Dk)x^k^K need not have 
Lipschitzian inverses. In Section |4l we apply this framework to the regularization of inconsistent 
common zero problems. Finally, Sections [5] and [6] address, respectively, applications to multivariate 
and univariate structured convex minimization problems. 

Notation. We denote the scalar product of a Hilbert space by (• | •) and the associated norm by 
|| • || . The symbols — and — > denote, respectively, weak and strong convergence, and Id denotes the 
identity operator. Let % and Q be real Hilbert spaces and let 2 H be the power set of %. The space 
of bounded linear operators from % to Q is denoted by IB (H,G)- Let A: % — > 2 H . We denote by 
ram4 = [u G % | (3 x G H) u £ Ax] the range A, by dom A = {x £ % | Ax / 0} the domain of A, 
by zer A = {x G % | G Ax] the set of zeros of A, by gra^4 = {(2, u) G % x H. | u G Ax] the graph 
of A, and by A^ 1 the inverse of A, i.e., the operator with graph {(u, x) G % x % | u G Ax]. The 
resolvent of A is J a = (Id + A)" 1 . Moreover, A is declared monotone if 



and maximally monotone if there exists no monotone operator B : % 2 H such that graA c 
graB ytz graA. In this case, J a is a nonexpansive operator defined everywhere on %. Furthermore, 
A is uniformly monotone at x G dom A if there exists an increasing function 0: [0, +00 [ — > [0, +00] 
that vanishes only at such that 



and A is couniformly monotone at u G ram4 if A 1 is uniformly monotone at u. The parallel sum of 

A and B : U -> 2 H is 



minimize 

xieHi,..., x m €H. 




(1.6) 



(V(x, u) G graA)(V(y, v) G gra^4) (x — y \ u — v ) ^ 0, 



(1.7) 



(V-u G Ax)(y(y, v) G graA) (x - y \ u — v ) ^ <p(\\x — y\\) 



(1.8) 



ADB = (A + B ) 



1 



(1.9) 
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The infimal convolution of two functions g and t from % to ]— oo, +00] is 

gUl: H -)• [-00, +00] : x^ inf (g(y) + £(x-y)). (1.10) 

yew 

We denote by To(^) the class of lower semicontinuous convex functions / :%—>■ ]— 00, +00] such 
that dom/ = {x G % | /(z) < +00} ^ 0. Let / G r (^). The conjugate of / is F (H) 3 f*:u^ 
sup x€n ((x I u) — f(x)), and / is uniformly convex at x G dom / if there exists an increasing function 
<ft : [0, +00 [ — > [0, +00] that vanishes only at such that 

(Vy G dom /)(Va G ]0, 1[) f(ax + (1 - a)y) + a(l - a)</>(\\x- y\\) < a/(s) + (1 - a)/(y). (1.11) 

For every x EH, f + \\x — - || 2 /2 possesses a unique minimizer, which is denoted by prox^x. We have 

pro X/ = J 9/ , where df: U -> 2 W : x ^ {u G ft | (Vy G H) (y - x \ u) + f(x) < f{y)} (1.12) 

is the subdifferential of /. Let C be a convex subset of ft. The indicator function of C is denoted 
by lc and the distance function to C by dc- The relative interior [respectively, the strong relative 
interior] of C, i.e., the set of points x G C such that the cone generated by — x + C is a vector 
subspace [respectively, closed vector subspace] of ft, by ri C [respectively, sri C] . See [U l46l for 
background on convex analysis and monotone operators. 



2 General algorithm 

We start with three preliminary results. The first one is an error-tolerant version of a forward- 
backward-forward splitting algorithm originally proposed by Tseng 1143 [ Theorem 3.4(b)]. 

Lemma 2.1 lfT3l Theorem 2.5(i)-(ii)] Let K be a real Hilbert space, let P : 7C — > 2 K be maximally 
monotone, and let Q : K. —■ K be monotone and x-Lipschitzian for some x £ ]0, +oo[. Suppose that 
zer (P + Q) / 0. Let (a n ) ne ^ (b n )n£N, cmd (c n ) ne N be absolutely summable sequences in K, let 
w G K, let e G ]0, l/(x + 1)1 let (7 n )neN be a sequence in [e, (1 — e)/x], and set 

For n = 0, 1, . . . 

s n = w n - 7„(Qtu n + a n ) 

Pn = JlnP s n + b n (2.1) 
In = Pn ~ ln{QPn + c n) 

_ w n+ i = w n - s n + q n . 

Then X^neN \\ w n — P n \\ 2 < +°° an( ^ there exists w G zer (P + Q) such that w n — w and p n — * w. 

Lemma 2.2 [8, Proposition 23.15(h) and 23.18] Let ft be a real Hilbert space, let A: % ->■ 2 n 
be a maximally monotone operator, let 7 G ]0, +oo[, and let x and r be in ft. Then J 7(r+J 4-i)X = 
x - j(r + J 7 -i A (7 _1 x - r)). 

Lemma 2.3 dH Proposition 2.8] Let ft and Q be two real Hilbert spaces, let E : H -> 2 n and 
F : Q -)■ 2 G be maximally monotone, let L G S (ft, £?), Zet z G ft, and let r £ Q. SetK = / H®Q, 

M : K -)■ 2 70 : (as, w) i-> (-z + Ex) x (r + F^v) 
S:K^K:(x,v)^(L*v,-Lx), 
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and 



'<$ = {x£-H\z£Ex + L*{F{Lx - r))} 
® = {v E £ | -r £ -L(E- l {z - L*v)) + F^v}. 



(2.3) 



Then zer (M + 5) is a closed convex subset oftyxT), and / 44> zer (M + S) / 44> :2> / 0. 

The following theorem contains our algorithm for solving Problem [Tj] and states its main asymp- 
totic properties. In this primal-dual splitting algorithm, each single-valued operators is used explic- 
itly, while each set-valued operators is activated via its resolvent. Approximations in the evaluations 
of the operators are tolerated and modeled by absolutely summable error sequences. The algorithm 
consists of three main loops, each of which can be implemented on a parallel architecture. 

Theorem 2.4 Consider the setting of Problem U . 1\ For every i 6 {1, . . . , m}, let (ai,i, n )neN> (bi,i, n )neN, 
and (ci i i in ) n6 N be absolutely summable sequences in Hi and, for every k E {1, . . . , K}, let (ci2,k,n)neN, 
{b2,k,n)nm, and (c 2 ,k,n)nem be absolutely summable sequences in Q k . Let x lfi e Hi, x mfi e H m , 
vi,o G Gh • • vk,o G Gk, let e e ]0, l/(/3 + let (j n ) n em be a sequence in [e, (1 - e)//3], and set 

For n = 0, 1, . . . 
For i = 1, . . . , m 

Sl,i,n = x i,n 7n [Ci%i,n ~\~ Xyfc=l ^ki^k^n ~H Q>l,i,n 
Pl,i,n = J-y n Ai{si,i,n + 7nZi) + h,i,n 



For k = 1, . . . , K 

S2,k,n = Vk, n ~ Jn(^D k 1 Vk t n ~ YrtLl ^ki^i,n + 02,/c, 
P2,k,n = S 2)k ,n ~ ln{rk + J^ 1 B k (in 1 s 2,k,n ~ rjfc) + &2,fc,n) 

Q2,k,n = P2,k,n ~ 1n(^D k 1 p2,k,n ~ YllLl ^kiPl,i,n + C2,k,n 
Vk,n+l = Vk,n — S2,k,n + </2,fc,n 

For i = 1, . . . , m 

Ql,i,n = Pl,i,n — In fCiPl,i,n + Ylk=l L*kiP2,k,n + 
%i,n+l — %i,n Sl,i,n ~i~ (?l,i,n- 



(2.4) 



Then the following hold. 



(i) (V*G {!,..., m})£ neN 



Pi, 



< +oo. 



(ii) (Vfc G {1, . . . , if}) E nGN [|«fc,n - P2,fc,n|| 2 < +oo. 



(hi) There exist a solution {x\, . . . ,x m ) to (II .2D and a solution (v±, 
following hold. 



,vk) to (11 .3D such that the 



(a) (V« G {1, . . . , m}) Zi - J2k=i L ti v k G AiXi + Qxi. 

(b) (Vfc e {1, . . . , if}) E^i ^fci^ " ^ G B k l7 "k + D^vj:. 

(c) (Vi G {1, . . . ,m}) x it n xj and p ljijn xj. 

(d) (Vfc G {1, . . . , if}) vjfe )n Vk~ and p 2jk ,n ~* 

(e) Suppose that, for some j e {1, . . . , m}, A,- or C 3 - is uniformly monotone at x]. Then Xj t . 

x] and pij tU ->■ xy. 
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(f) Suppose that, for some I G {1, ... ,K}, B\ or D\ is couniformly monotone at v\. Then 

%n -> W and p 2 ,z,n ->■ W- 



Proof. Let us introduce the Hilbert direct sums 

U = Ui®---® U m , Q = Gx® ■■■®Gk, and K = H@G, 



(2.5) 



and let us denote by a; = (£i)i<ji<j m and t> = (vk)i^k<:K generic elements in ~H and Q, respectively. 
We also define 



A: H ^ 2 n : x ^ X A 



1 1 



i=i 



C : T-L ^ H : x ^ (CiXi)i^ r 
E = A + C 

/ m 

L : H — > : a; i— )• I ifci^ 



and 



i=l 



S : £ -»• 2 e : « ^ X -Bfc^fe 
fc=l 
if 

Z> : 9 -»• 2 6 : « ^ X £>^fc 
fe=l 



(2.6) 



It follows from HI Proposition 20.22 and 20.23, Corollaries 20.25 and 24.4(f)] that A, B, C, D, E, 
and F are maximally monotone. Moreover, L G "3(1-1,0), L* : Q — > H : v h-> (J2k=i ^ki v k)^Km> 
and 



if m 



(VxeU) \\Lx\\ 2 = ^2 L kiX t 

k=l 1=1 



«S A||x| 



(2.7) 



Next, we set 



' M: K^2 K : (x,v) H- (-z + Fx) x (r + F~ x i>) 
P : /C ->■ 2 7C : (a?, v) (->■ (-z + Ax) x (r + B _1 i;) 
Q : K->K : (x,v)^ (Cx + L*v, D^v - Lx) 
R.K^K . (x,v) i — ^ (Cx, LT 1 ??) 

^5: K^K: (x,v) (X*v,-Lx). 



(2.8) 



Note that 

zer(P + Q) = {(x,u) G "ft © 3 | z - L*v G Ax + Cx and Lx — r £ B~ 1 v + D~ 1 v}. (2.9) 

Furthermore, in view of [51 Propositions 20.22 and 20.23], P is maximally monotone, and 
Lemma [2721 and [J8l Proposition 23.16] yield 

(V7 G ]0, +oo[)(Vx G ?i)(Vv G Q) J lP (x,v) = (JjaAx! +721), . . . , J 1 A m (x m + jz m ), 

Vl - 7(n + Jj-ibA^ 1 ^ ~ r l))' • •• > V K - l( r K + J^-i Bk {1~ X VK ~ r K ))j- (2.10) 



On the other hand, since C and D~ l are monotone and Lipschitzian with, respectively, constants 
H = maxi^j^ m /ij and v = max^^A'^/o R is monotone and Lipschitzian with constant max{^, u}. 
In addition, it follows from (TU Proposition 2.7(ii)] and (12771) that S 6 IB (JC, 7C) is a skew (hence 
monotone) operator with \\S\\ = \\L\\ ^ \/\. Altogether, since Q = R+ 5, we derive from (II. ID that 



P is maximally monotone and Q is monotone and /3-Lipschitzian. 



(2.11) 
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Let us call *p and 2) the sets of solutions to (11.2ft and (11.31) . respectively. It follows from (12.61) that 

to = {x€H\z€Ex + L*(F(Lx - r))} 

1 2) = {v e g \ -r e -L(E-\z - L*v)) + F^v). 



Hence, since *JJ / by assumption, we deduce from Lemma [231 that 

/ zer(M + S) = zer(P + Q) c x 2). (2.13) 
Thus, to solve Problem ll.li it is enough to find a zero of P + Q. For every n G N, let us set 



(2.14) 



w n = 


(^1,71) • • 


• j ■Em,ni ^l,ni • • • 


VK,n) 


Sn — 


( s l,l,ri) • • 


• > ^l,m,nj S2,l,n) ■ 


■ ■ , s 2,K,n) 


Pn = 


(Pl,l,n> • • 


• > Pl,m,n> P2,X,n > 


■ ■ ■ ,P2,K,n] 


Qn = 


(?l,l,ri) • • 


■ > Ql,m,m Q2,l,m ■ 


■ ■ , Q2,K,n) 



and 



a, 
b„ 



(ai 

,l,rt) • • • > ^l,m,rt) ^2,1, n> • • • > Q j 2,K,n) 
(bl,l,n, ■ ■ ■ j bl,m,n, ~ 7n&2,l,nj ■ ■ ■ j ~ 7n^2,E",n) 
( c l,l,n> • • • > c l,m,n> c 2,l,n, ■ ■ ■ , C2,K,n)- 



(2.15) 



Then, using (12.61) . (12.81) . and (12.101) . we see that (12.41) reduces to (12. 1 D . Moreover, our assumptions 
and (I2.5D imply that (a n ) rag N, (&n)neN> an d (c n ) n£ N are absolutely summable sequences in /C. Hence, 
it follows from (12.111) . (12.131) . and Lemma [27T1 that XmeN II "'n — Pnll 2 < +°° an d that there exists 



id G zer (P + Q) such that w n w and p n — * w. Upon setting = (x±, . . . , x m , v\, . 



and 



appealing to (12.5ft and ( 12.91 ), we thus obtain assertions [(1)1 1 (ii) [ and | (hi) (a)]4(iii) (d) 
| (hi) (e)[ Let us introduce the variables 

K 

Si i n — %i 



(Vie {l,...,m})(Vn G N) 



,n ~~ 7n ( CiXi t n + L* ki Vk t n j 
^ fc=l ' 

Pl,i,n = Jy n Ai(si,i,n + 7n z i) 



(2.16) 



and 



(Vife G {l,...,if})(Vn G N) 



7n ( r-fe + J -i Bfe (7 n 1 s 2 ,fc,n - r fc ; 



•S2,fe,n — ^fc,n — 7n ( D k ^fc.n ~ ^ L ki Xi^ n 
P2,k,n = S2,k,' 

V 

It follows from (12.4ft that 

(Vi G {1, . . . ,m})(Vn G N) ||s liiin - Si,i, n || = 7n||ai,i,n|| < ^IK.i.nll- 
Hence, by virtue of the nonexpansiveness of the resolvents [31 Proposition 23.7], we have 



(2.17) 



(2.18) 



(Vi G {1, . . . ,m})(Vn G N) \\pi,i, n -Pi,i,n\\ = ||^ 7n Ai(si,i,n + InZi) + bx^ n -J lnAi (sx^ n + ^ n Zi)\\ 

^ 11^1, i,n •Si,);, n|| ~i~ ||^l,j,n|| 

^/3 -1 ||ai,i,n|| + ||fcl,i,n||- (2.19) 
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In turn, since, for every i G {1, . . . , m}, (ai 5 j in ) ng N and (fri^rOneN are absolutely summable, we get 



(Vi G {1, ... ,m}) si,i, n - si,i jn ->• and p ljijn - ->■ 0. 
Likewise, we derive from Q2.4D and (I2.17D that 

(Vk G {1,...,K}) S 2 ,k,n ~ S 2 ,k,n ~> and p 2 ,fc,n - P2,fc,n ~>" 0. 



On the other hand, we deduce from |(iii) (a)| that 

(Vi G {1, . . . ,m})(3ui G Hi) Ui e A{xl and ^ = Uj + ^ + Cj. 



K 



k=l 



and from | (hi) (b)| that 

(\fk G {1, . . . ,K}) v^e B k ( L u xl - r k - D^Vkj. 

^ i=i ' 

In addition, (12.161) yields 

~ K 

(Vi G {1, . . . , m})(Vn G N) — — Vl^hlL - ^ L* ki v kjn - CiX i>n + Zi G Ajp ljijn , 

7n 

while (IZTTl) yields 



fc=i 



7n 



Now, let us set 



(Vra G N) S n = y^(- + u k \ \\v kjn - p 2 ,k,n\\ [|P2,fc,n - '"fell and (Vi G {1, . . . , m}) 
fc=l ^ £ ' 



A' 



It follows ftomlTOl Iffifl |(iii) (c)j [Gii)(d)I (12301) . and (I2T2TT) that 

£ n — >■ and (Vi G {1, . . . , m}) a^ n — > 0. 



k=l 



(2.20) 



(2.21) 



(2.22) 



(2.23) 



(2.24) 



m v 

(VfcG{l,...,ET})(VnGN) p 2 , M G £ fc ( ~ P2 ' k ' n + ^ L^n - r k - D~ k \ Kn j . (2.25) 

8=1 ' 



(2.26) 



(2.27) 



Using the Cauchy-Schwarz inequality, the Lipschitz-continuity and the monotonicity of the operators 
(Ci)i<i<mj (12.22ft . (I2.24D . and the monotonicity of the operators (^4j)i^ m , we obtain 



(Vi G {1, . . . ,m})(Vn G N) a^ n + ( x i}n - x 



K 



L li( v k ~ V k ,n) 



k=l 
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> \\Pl,i,n ~ Xi,n\\{£ \\Pl,i,n 
K 



K 



J2 L lt( v k - Vk,n) 



k=l 



%i\\ ~i~ \\CiXi^ n CjXj||j + ^ Pl,i,n %i,r 
^2 L *ki(Vk-Vk,n) 

n ~ %i,n\\ ( e 1 ||Pl,i,n ~~ x i\\ + ||Cj^i,n ~~ Cj^iH) + ( Pl,i,n ~ %i ^*ki{ v k ~ v k,r 

\ k=l 



^ ( Pl,i,n %i 



Pl,i,n %i 



K 



In 

•Ei,n Pl,i 



+ ^ ] L\i(vk — Vk,n) \ + (Pl,i,n ~ x i,n \ C{X{ — CiXi^ n ) 
k=l ' 
K K 

~ L* k i v k,n ~ CiXi^n + L* k i v k + Cf 



In 



k=l 



k=l 



~t~ {^i,n 3Ci | CiXi^ n CiXj^ 



Pl,i,n %i 



%i,n Pl,i,' 
In 



K 



y ] ^hi v k,n ~ CiXi jTl + 



Zi llj 



k=l 



{^i,n 3Ci | CiXi^ n CiXj^ 



^ \ P~l,i,n ■£» 
^0. 



2-1,71 Pl,i,r. 
In 



K 



y ] ^ki v k,n ~ CiXi^ n + Zj ) — U, 
fc=l 



(2.28) 
(2.29) 
(2.30) 



On the other hand, since the operators (D fc ^l^ts^ are Lipschitzian and monotone, and since the 
operators (B k )\^ k ^K are monotone, we deduce from (I2.26D . (I2.23D . and (I2.25D that 



(V/ G {1. . . . , K})(\/n £ N) <5 n + J^ 



8=1 



2-2, n 



k=l 



K 

>E 

fc=l 

-E 

it=i 



^fc,n — P2,k,n 
In 

Vk,n—P2,k,n 



D k 1 p2,k,n ~ D k 1 V k>n + ^ L 



kiy^i.n %i) 



i=l 



P2,k,n ~ V k 



In 



i=l 



+ ^2L ki x itn -r k -D k l v kiTl I - I ^ L fci Xj - r fc - D k l v k 



P2,k,n~Vk 



K 



+ ^2 (D k 1 p2,k,n ~ D k 1 V k | p 2 ,fc,n ~ Vk) 



(2.31) 



fc=l 

VLn ~ P2,Ln 



rn \ / Til \ 

— + ^ L H Xi > n ~ r l- D l ly l,n ) ~ \ ~ r l~ D l lW l ) 

ln i=l ' V i=l / 



P2,Ln ~ V\ 



+ (D t 1 p 2 ,l,n ~ D t 1 Vl | p 2 ,l,n ~ Uj) 



(2.32) 



Vl,n ~ P2,l,n 
In 



i=l 



+ ^ LiiXi )tl -n-D x 1 vi iTl ) - I y~] LjiXj -ri- D l 1 v l 



i=l 



P2,l,n ~ Vl 



>0. 

We consider two cases. 



(2.33) 
(2.34) 



• If Aj is uniformly monotone at x], then, in view of G2.29D , (12.221) . (I2.24D , and (11.8ft , there 
exists an increasing function 0^ : [0, +00 [ — > [0, +00] that vanishes only at such that 



(Vn E N) aj )7l + ( Xj jn - xj 



K 



fc=l 



L *kj( v k ~ V k ,n) ) > <pA 3 (\\Pl,j,n ~ Xj 



(2.35) 



Combining (12341) . (12301) . and (12351) yields 



mm, K > 

(Vn G N) 5„ + ^ + ^ ( - x7 y^£to(p2,fc,n - «fc,n) ) ^ 0Aj(llPi^>~^7ll)- ( 2 - 36 ) 

i=l i=l \ k=l ' 

It follows from J2I27D , RiiTl fCiii) Cc) L (IZ2TT) . and d Lemma 2.41(iii)] that <£ Aj ; (||pij>-Sj||) -> 
and, in turn, that Pij, n — >• ^J- In view of [(I)] and (I2.20D . we get pij, n — > ~j and Xj >n — > x]. 

• If Cj is uniformly monotone at x], then we derive from (12.34D . (12.28D . and (I2.30D that there 
exists an increasing function ipCj ■ [0, +00 [ — > [0, +00] that vanishes only at such that 

K 



(Vn EN) 5 n + Y (*i,n + Yl 



Xi, n %i 



k=l 



> (f> Cj (\\x j!n -x]\\). (2.37) 

This implies that (j>cA\\%j,n — — ^ and hence that x^ n — >■ xj. Finally, [(1)1 yields p%j,n ~^ ~x]- 



(iii) (f) 1 We consider two cases. 



• If B\ is couniformly monotone at v\, then (I2.33D . (12.23ft . and (I2.25D imply that there exists an 
increasing function <fi B -i : [0, +00 [ — > [0, +00] that vanishes only at such that 



(Vn E N) S n + 



8=1 



X in Xn 



K 



J2 L ki(P 2 > k > n ~ Vk - 



k=l 



(~ m \ / m 

— — — + Lux i}n -ri - p^v^n ) - [yz Li 
Tn • -, / V • 1 

i=i ' x i=i 



Xi-ri - D, \ 



P2,l,n-Vl 

(2.38) 



Combining this with (12.301) yields 

mm, K \ 

(Vn E N) 5 n + Y ai > n + Y\ Xi > n ~ Wi Y L *ki^ 2 > k ' n ~ v k,n)) > (||P2,J,n-Wll)- (2.39) 

i=l i=l ^ fc=l ' 

Hence, using (lZ271) ,[0PT}[(nTnc)| (12^TD . and © Lemma 2.41 (iii)], we get # B -i ([|p2,j,n-Wl|) -> 
and, in turn, P2,z,n — ► W- Using to (12.211) and |(ii)[ we conclude that P2,i, n — ^ W an d Vj,n — ► W- 

• If Di is couniformly monotone at vj, then it follows from (12.32D and (I2.34D that there exists an 
increasing function 4> D -i : [0, +oo[ — > [0, +oo] that vanishes only at such that 



m 



8=1 



K 



(Vn E N) <5 n + Y \ X i,n ~ X i Y L *ki(P2,k,n ~ Uft)/ > {D l 1 p 2 ,l,n ~ Dj 1 V i \ p 2 ,l, n ~ v{) 

(2.40) 



k=l 



> <t> D --L{\\P2,l,n ~ Vi\ 
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Thus, (12301) yields 

m m 

(VnGN) S n + ^2a i: n + Y^ 

i=l i=l 

and we conclude as above. 



K 



k,n ^k,r 



k=l 



52,/,n-W||), (2.41) 



□ 



Remark 2.5 When m = 1, Theorem 12.41 specializes to []T9l Theorem 3.1]. Our proof of Theo- 
rem [23[UH(nIlIlll hinges on a self-contained application of Lemmas 12.31 and [2TT1 in the primal-dual 
product space 7C of (12.5ft . Alternatively, these results could be obtained as an application of [fT9l 
Theorem 3.1] using the product space "K of (I2.5D as a primal space. This strategy, however, would 
not allow us to recover the strong convergence results of Theorem HjJmJCeJ 



Remark 2.6 It follows from the Cauchy-Schwarz inequality that, for every (xj)i^ m G 0™ 1 1-Li, 



K 



k=l i=l 



2 K 



1 1 -^2 



fc=l V i=l 



2 A" 



E EM 2 E 



fc=i v i=i 



i=i 



(2.42) 



Hence, in general, one can use 

A = EfeLi EI=i II 2 in (El])- However, as will be seen in subse- 
quent sections, this bound can be improved when the operator L of (12.61) has a special structure. 



In the remainder the paper, we highlight a few instantiations of Theorem 12.41 that illustrate the 
variety of problems to which it can be applied and which are not explicitly solvable via existing 
techniques. 



3 Inclusions involving general parallel sums 

The first special case of Problem 11.11 we feature is an extension of a univariate inclusion problem 
investigated in H19L which involves parallel sums with monotone operators admitting Lipschitzian 
inverses. In the following formulation, we lift this restriction. 

Problem 3.1 Let Jibe a real Hilbert space, let K\, K 2 , and K be integers such that < K\ ^ K 2 ^ 
K ^ 1, let z G %, let A : % — > 2 H be maximally monotone, and let C : Ti — > Ti be monotone and 
^-Lipschitzian for some fi G [0, +oo[. For every integer k G {1, . . . , K}, let Gk be a real Hilbert space, 
let r k G Gk, let B k : Gk ->■ 2 Gk and S k : Gk -> ^ Gk be maximally monotone, and let L k G S (H,Gk); 
moreover, if if i + 1 < A; ^ K 2 , S k : G k — ^ Gk is -Lipschitzian for some € [0,+oo[, and, if 
K 2 + 1 < k < K , : Gk ^ Gk is Pk -Lipschitzian for some f3 k G [0, +oo[. It is assumed that 



A 



l + ^||L fc || 2 >0, (3.1) 



k=l 



(3 = max{n,/3 Kl+1 , . . .,(3 K } + 
and that the inclusion 

find x en such that z G Ax + ^ L* k ((B k □ S k )(L k x - r k )) + Cx (3.2) 



K 



k=l 
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possesses at least one solution. Solve (13.2ft together with the dual problem 



find v\ G Q\ , . . . , Vk G Gk such that 



(Vfce{l,...,iT}) -r k G -Lfe^ + C)- 1 ^-^^)) +£^ + <S fc - 1 ^. (3.3) 

Proposition 3.2 Consider the setting of Problem \3.1\ Let (ai i i )n ) ne Nj> (&i,i,n)neN> (ci,i,n)neN ^ e 
absolutely summable sequences in %. For every integer k G {1, . . . , -KT}., Zet (c^fc^neNj (&2,fc,«)«eNj> and 
(c2,fc,n)neN be absolutely summable sequences in Gk', moreover, if 1 ^ k ^ K\, let (6i ) fc+i, n ) n eN be an 
absolutely summable sequence in Gk, and, if K\ + 1 ^ k ^ K 2 , let (ai,fc+i, n )neN (ci,fc+i,n)neN be 
absolutely summable sequences in Gk- Let xq G yi^ G Gi, ■ Uk 2 ,o e fe 2 ^ u i,o 6 £i> an ^ 
vk,o G £7iG Zet e G ]0, l/(/3 + let (7„) ne N be a sequence in [e, (1 - and set 

For n = 0, 1, . . . 



Sl,l, n = X n - ^ n {Cx n + ^CfcLl £jfc w fc,n + a l,l,n) 
Pl,l,n = ,l,n 

IfK 1 ^0,fork = l,...,K 1 

51, k+l,n = Uk,n + lnV k) n 
Pl,fc+l,n = Jj n S k s l,k+l,n + ^l,/c+l,n 

52, k,n = Vk,n ~ ln(yk,n ~ Lk%n + a2,fc,n) 

P2,k,n = S 2) k,n ~ 7n( r fc + ^ ^ B k ("Cn ls 2,fc,n ~ »"fc) + &2,fc,7i) 
<72,fc,7i = £>2,fc,n ~ 7n (Pl,fc+l,n ~ LkPl,l,n + C2,fc,n) 
. ^fc,n+l = v k,n — S2,k,n + 92,fc,n 

IfK 1 ^K 2 ,fork = K 1 + l,...,K 2 

51, k+l,n = Uk,n ~ ln(S k yk,n ~ V k ,n + a l,fc+l,n) 
Pl,k+l,n = Sl,k+l,n 

52, k,n = v k,n ~ ln(yk,n ~ Lk%n + a2,fc,n) 
ln{rk + J^ B ,{ln l S2,k,n 



P2,k,n = S 2 ,k,n ~ 7n( r fc + J ^ B k (">n 1 s 2,k,n ~ rk) + b 2 ,k,' 
Q2,k,n = P2,k,n ~ ln{Pl,k+l,n ~ LkPl,l,n + C 2t k,n) 
Vk,n+1 = Vk.n — S2,k,n + Q2,k,n 

r- , ■ : ■ K 



IfK 2 ^K, fork = K 2 + l,....,, 

S2,k,n = Vk,n ~ Jn(S k ~ + «2,fc,n) 

P2,k,n = S 2 ,k,n ~ 1n[Tk + J^B k (7n 1 s 2,fc,n ~ ?*jfc) + &2,fc,ra) 
92,fc,n = P2,fc,n ~ 7n(5' fc P2,k,n ~ LkPl,l,n + C2,fc,n) 
Wfc,n+1 = Vk,n — S2,k,n + Q2,k,n 
?l,l,n = Pl,l,n - 7n(Cpi,l,n + SfcLl L* k P2,k,n + Cl,l, n ) 
= iCn — s l,l,n + 91,1,71 

7/^1/0,^^ = 1,...,^! 

9l,Jfe+l,n = Pl,fc+l,n + 1nP2,k,n 
Vk,n+l = J/fc,n — s l,Jfc+l,Ti + Ql,k+l,n 

IfK 1 ^K 2 ,fork = K 1 + l,...,K 2 

= Pl,fc+l,n — Jn{SkPl,k+l,n ~ P2,k,n + Cifi+^n) 
- m, — e, , . - J- n, , . -, 



9l,fc+l,n 

Vk,n+1 = Uk,n ~ Sx,k+l,n + 9l,fc+l,r 



(3.4) 



Then the following hold for some solution x to (13.2ft and some solution (v\,..., vk) to (13.31) 



(i) x n ^ x and (Vk G {1, . . . , K}) u&. 



''A.:- 
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(ii) Suppose that A or C is uniformly monotone at x. Then 

(iii) Suppose that, for some I G {1, . . . , K}, Bi is couniformly monotone at vj. Then v^ n — > v\. 

(iv) Suppose that K2 / K and that, for some I G {K2 + 1, . . . , K}, Si is couniformly monotone at vj. 
Then v^ n -)• vj. 

Proof. We assume that K 2 ^ and consider the auxiliary problem 
find x G H, yl G Q\, . . . , G Qk 2 such that 

K 2 K 
z G Ax + Y^L* k {B k {L k x-yk-r k )) + ^ ^((^fe □ S k ){L k x - r k )) + Cx 

k=l k=K 2 +l 

G S\y\ — Bi(L\x — y\ — r\) 
k G Sk 2 iJk^ - B K2 (L K2 x - yK~ 2 - rx 2 ) 



(3.5) 



together with the dual problem (I3.3D (if K2 = 0, (I3.5D should be replaced by (I3.2D and the resulting 
simplifications in the proof are straightforward) . Let us show that solving the primal-dual problem 
(13.5D / (I3.3D is a special case of Problem ll.il with 



m 


= ^2 + 1 


u x 


= H 




= A 


Ci 


= C 


Ml 


= V 


zT 


= X 


3 


= z, 



(yke{l,...,K 2 }) { 



Ak+i 



Ck 



+1 



(S k , if lO^i; 

jo, if ifi + 1 < fc ^ K 2 

jo, if 1 < k ^ iTi; 

\s fcl if^i + 10^ 2 

Jo, if 1 < lb < Jfi; 

I if Ki + 1 < k «c K 2 



(3.6) 



and 



(VfcG {!,...,#}) 4 



^fc+i = 0, 



'{0}-\ if 1 < k s$ K 2 ; 



if if 2 + 1 s£ k < if 



0, if 1 ^ Jfe ^ if 2 ; 

if K 2 + 1 < fe < K 



Vk+l = < 

Lki = L k 

(Vt€ {2,..., ^2 + 1}) L fci 



(3.7) 



-Id, if i = fc + 1; 
0, otherwise. 



First, we note that, in this setting, (II .2D reduces to (I3.5D . and (II .3D to (I3.3D . Now define % and 
Q as in (1231) . let x £ 7-1, let (yk)i^k^K 2 6 0^^, set = (x,yi, . . . ,y K2 ) G set 

Then, using the Cauchy-Schwarz 



2/ 



( yi ,...,y^ 2 ,0,...,0) G 0, and set A = 1 + £fii ||L 
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inequality in 

K n 



fc=l i=l 



\\(L k x)x^ k< K 2 - y\\ 2 < (||y|| + \\(L k x) 1<k<K2 

2 



< llvll + 



^2 
\ k=l 



k\\ 2 \\x 



,) <( 1+ |W). 



|y|| 2 + IMI 2 )=A£| 



(3.8) 



1=1 



Thus (11 . ID is a special case of specializes to (I3.1D . On the other hand, by assumption, (I3.2D has a 
solution, say x. Therefore, there exist v\ & Q\, . . . , vk 2 G <5jf 2 such that 



K 2 K 

z e Ax + Y^L* k v k + Yl L%((B k US k )(L k x-r k ))+Cx 

k=l k=K 2 +l 

(Vke{l,...,K 2 }) v k e (B k aS k ){L k x-r k ). 



Therefore, in view of ( I1.9D , there exist y\ e Q\, . . . , yx 2 G Qk 2 such that 

' K 2 K 

z £ Ax + Y^L* k v k + Yl L* k ((B k nS k )(L k x-r k )) +Cx 

k=l k =K 2 +l 

(Vk £ {1, . . . , K 2 }) y k G S k x v k and L k x - y k - r k G B^vj,, 
which implies that 



K 2 K 

eAx + ^L* k v k + Y L* k {{B k USk){L k x-rk)) +Cx 

k=l k=K 2 +l 

G {1, • • • , K 2 }) v k G S k yk and v k G B k (L k x -y k - r k ), 



and therefore that 

K 2 K 

e Ax + Y^L* k {B k {L k x-y k -r k )) + ^ ^((^ D S k ){L k x - r k j) + Cx 

k=l k=K 2 +l 

(Vk G {1, . . . ,K 2 }) e S fe i/ fc - B k (L k x - y k - r k ). 
This shows that (I3.5D possesses a solution. Next, upon defining 



(Vn e N) and (Vfc G {1,...,K 2 }) { 



(3.9) 



(3.10) 



(3.11) 



(3.12) 



(3.13) 



we see that (|2~4l) specializes to ([3~4l) . Hence, in view of (l3T6l) -([3~7l) and Theorem [2.4ltiii) (a)f[(iii) (d)| 
there exist a solution (x, yl , . . . , yx^) to (13.5ft and a solution (uf, . . . , vk) to (13.31) such that 



Xk+l,n ~ 


~ Uk,n) 




0>l,k+l,n 


= 0, 


if 1 < fc < ifi; 


&l,fc+l,n 


= 0, 


if + 1 it < if 2 


^Ci,fc+l,n 


= 0, 


if 1 < fc < ifi, 



x and (Vk E {1,... ,K}) v k>n 



Vk, 



(3.14) 
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with 

K 



E r *—^ A - t „- / WI ^ n rzl N \L k x-y k -r k GB k 1 v k 
L k v k £ Ax + Cx, (Vfc G {1,... ,K 2 }) <_ 



k=l 



and (V& G {A^ + 1, . . . , A"}) L k x - r k G S" 1 ^ + (3.15) 
Since the strong convergence claims |(ii)f|(iv)| are immediate consequences of Theorem |2. 4][hi)(e)T - 



(iii) (f) [ it remains to show that x solves (I3.2D . We derive from (13.15ft that, for every k G {1, . . . , K2}, 
L k x-yk-r k G B^Vk and Vk G S7 1 Vfc, and, for every A; G {A" 2 +l, • • • , K}, L k x-r h G B^v^+S,: 1 !^. 
Altogether, 

(VA;G{l,...,if}) L fc x-r fc G (fi^ 1 + S^ 1 )^ (3.16) 
and, therefore, 

K K K 

J2LtWeY, L U( B k 1 + Sk 1 y 1 (L k x-r k )) =Y J Ll{(B k DSk)(L k x-rk)). (3.17) 

k=l k=l k=l 

Thus, since (13.15ft also asserts that z — J2k=i ^V^k S Ax + Cx, we conclude that x solves (13.21) . □ 

Remark 3.3 Problem [3.1l encompasses more general scenarios than that of |fT9l] , which corresponds 
to the case when K\ = K2 = 0, i.e., when all the operators {DZ )x^ k ^K are restricted to be Lips- 
chitzian. This extension has been made possible by reformulating the original primal problem (13.21) . 
which involves only one variable, as the extended primal problem (13.51) . in which we added K2 
auxiliary variables. We also note that Algorithm (13.41) uses all the single-valued operators present in 

-In 



ProblemED] including {S k ) Kl +i^ki:K 2 and (S k )K 2 +Kk^K, through explicit steps. 



4 Relaxation of inconsistent common zero problems 

A common problem in nonlinear analysis is to find a common zero of maximally monotone operators 
A and (B^i^^k acting on a real Hilbert space H (l6l[22l[32]], i.e., 

K 

find x G H such that G Ax H O B k x. (4.1) 

k=l 

In many situations, this problem may be inconsistent (see H18I1 and the references therein) and it 
must be approximated. We study the following relaxation of (14.1ft . together with its dual problem. 

Problem 4.1 Let T-L be a real Hilbert space, let AT be a strictly positive integer, let A: % — > 2 n be 
maximally monotone, and, for every k G {1, . . . , K}, let Sk - H. — > 2 H be a maximally monotone 
operator such that SZ 1 is at most single-valued and strictly monotone, with SZ = {0}. It is 
assumed that the inclusion 

K 

find x G U such that £ Ax + ^2(B k D S k )x (4.2) 

k=l 
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possesses at least one solution. Solve (I4.2D together with the dual problem 
find u\ G H , . . . , uk S T~i such that 



(Vfc G {1,...,K}) G -A-^-^w) + ^ + ^ (4.3) 

^ z=i ' 



First, we justify the fact that (I4.2D is indeed a relaxation of (I4.1D . 



Proposition 4.2 In the setting ofProblem \4.1\ set Z = (zer A) n PlI—i ^erB k and suppose that Z ^ 0. 
Then the set of solutions to the primal problem (14. 2D is Z. 

Proof. It is clear that every point in Z solves (I4.2D . Conversely, let x be a solution to (I4.2D and let 
z e Z. We first note that the operators (-B& □ fife are at most single-valued. Indeed, let k G 

{1, . . . , K} and let and (y, g) be in gm(B k □ Then we must show that p = q. We have p = 
(B k U S k )y ^ y G B~ 1 p+ S^ l p 44> y- S^ l p G Likewise, y-S^q G and, by monotonicity 

of B k , -(p - q | S^p - S k x q) = (p-q \ {y - S^p) - (y - S^q)) ^ 0. Consequently, by strict 
monotonicity of S^ 1 , (p — q \ S^ l p — S^ l q) = and p = q. Hence, since x solves (I4.2D , there exists 
{pk)o^K G n K+1 such that 

K 

Y^Pk = 0, Po^Ax, and (Vfe G {1, . . . , K}) p k = {B k US k )x. (4.4) 

fc=0 

Therefore, we have 

Po G Ax, G Az, and (Vfe G {1, . . . , K}) p k G B k (x - S^Pk) and G B k z, (4.5) 
and, by monotonicity of the operators A and (B k )i^ k ^K, 

(x-z\ Po )^0 and (Vfc G {1, . . . , K}) (x - S~ l p k - z | p h ) > 0. (4.6) 
Hence, since X^fcLoP* = 0> ^ follows from the monotonicity of the operators (S k ~ 1 )i^ k <^K that 

K 



fe=i 



fc=0 fc=l 
K 

= {x - z I po) + ^2 {x - S k x p k -z\pk) 

k=l 

> 0. (4.7) 
Thus, (p* - | S^p k - S^O) = and, therefore, 

(Vfc G {1, . . . , if}) (p fe - | V - S^O) = 0. (4.8) 

The strict monotonicity of the operators (S^ )i^ k <^x implies that for every k G {1, . . . , K} p k = 0, 
i.e., x G B k l p k + S~V = B^0 + S^O = B^0. In turn, p = -£f=iPfc = 0, i.e., x G A^O. 
Altogether, x G Z. □ 
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Remark 4.3 Suppose that in Problem |4. II we set, for every k G {1, . . . , K}, Sk = T/T^d where 7^ G 
]0, +00 [, i.e., Bf.nSk = Jk Bk is the Yosida approximation of Bk of index jk ttH Proposition 23.6(ii)]. 
Then (I4.2D reduces to the setting investigated in |fT7[ Section 6.3], namely 



A' 



find x G U such that Oeii + ^ 7fe 5 fc ; 



(4.9) 



fc=i 



which itself covers the frameworks of IfTOl [T8l [531 [37ll and the references therein. In this case, 
Proposition 14.21 specializes to IfTTl Proposition 6.10]. Now let us further specialize to the case when 
n = R N , A = 0, and 



(Vke{l,...,K}) { 



Ik = 1 
Bk : x 1— > 



span{u fe }, if (x | u k ) = p k ; where < 

0, if (x I ujt) / 



u k G 

= 1 (4.10) 



Then (14. ID amounts to solving the system of linear equalities 
find x eR N such that (VA; G {1, . . . , K}) (x \ u k ) = p k , 

whereas (I4.2D amounts to solving the least-squares problem 

m 

minimize |(x | u^) — Pk\ 2 ■ 

~/-TO)iV ^- ^ 



(4.11) 



(4.12) 



fc=l 



The idea of relaxing (I4.11D to 04.12D is due to Legendre [31] and Gauss [26]. 



To solve Problem |4. 11 we use Proposition [JO] to derive the following algorithm. 

Proposition 4.4 Consider the setting of Problem \4.1\ Let (&i,i, n )neN and, for every k G {1, . . . , K}, 

{h,k+T-,n)nm and {b2,k,n)nen be abs olutely summable sequences in U. let x G U, (yk,o)i ^k£K £ 7~L K , 
{vkfl)i<:k^K G % K , and e G ]0, l/(s/K + 1 + 1) [, let (7„)„ e N be a sequence in [e, (1 - e)/y/K + 1], and 
set 

For n = 0, 1, . . . 

Vl,l,n = J-r„A[x n - 7„ Ylk=l v k,n) + h,l,n 

Fork = l,...,K 

Pl,k+l,n = J^ n S k {Uk,n + 1nVk,n) + ^l,fc+l,n 
S2,k,n = Vk,n ~ 1n(Uk,n ~ X n ) 

P2,k,n = S2,k,n ~ In (J~f- x B k (in ls 2,k,n) + &2,fc,n) 
Vk,n+l = Vk,n — S2,k,n + P2,k,n ~ 1n(Pl,k+l,n ~ Pl,l,n) 
X n +1 = Pl,l,n + In Ylk=l( v k,n ~ P2,k,n) 

For k = 1, . . . ,K 

[ Uk,n+1 = Pl,k+l,n + 1n(P2,k,n ~ V k , n ) 

Then the following hold for some solution x to (I4.2D and some solution (W[, . . . , Uk) to (I4.3D . 

(i) x n -± x and (VA; G {1, . . . , K}) v k>n Vk- 

(ii) Suppose that A is uniformly monotone at x. Then 



(4.13) 
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(iii) Suppose that, for some I G {1, . . . , K}, B[ is couniformly monotone at v\. Then v% n — > v\. 

Proof. Problem 14.11 is a special case of Problem 13.11 with K\ = = K, z = 0, C = 0, /i = 0, 
/3 = y/K + 1, and (Vfc G {1, . . .,K}) Q k = H, L k = Id , and r k = 0. In this context, (|3~4b can be 
reduced to (14.13D , and the claims therefore follow from Proposition 13.21 □ 

Remark 4.5 For brevity, we have presented an algorithm for solving Problem 14.11 in its general 
form. However, if some of the operators (S k )i<^k^K or their inverses are Lipschitzian, we can apply 
Proposition 13.21 with K% ^ K and/or K2 7^ K to obtain a more efficient algorithm in which each 
Lipschitzian operator is used through an explicit step, rather than through its resolvent. 



5 Multivariate structured convex minimization problems 



We derive from Theorem 12.41 a primal-dual minimization algorithm for multivariate convex mini- 
mization problems involving infimal convolutions and composite functions. 

Problem 5.1 Let m and K be strictly positive integers, let (Hi)i^i^ m and (Gk)i^k^K De real Hilbert 
spaces, let (Mi)i^m G [0,+oo[ m , and let (v k )i^K G ]0,+oo[ x . For every i G {l,...,m} 
and k G {1, . . . ,K}, let hi: Hi — > M be convex and differentiable and such that V/tj is /Xj- 
Lipschitzian, let /, G T {Hi), let g k G T (g k ), let 4 G T (g k ) be l/V fc -strongly convex, let z { G Hi, 
let r k G and let L k i G t B(Hi.Q k ). Set 5 = max] max itj, max v k \ + VX > 0, where 



A G 



sup^m i n^ip^! X^feLi II XXi -^fci^ill 2 ' +°° > and assume that 

K /in 



(Vi G {1, . . . ,m}) z, 
Solve the primal problem 



G nmfdfi + J2 L h (dg k Udl k ) o ( £ L fcj - • -r*) + . (5.1) 
^ fe=i ^i=i ' ' 



minimize V}/;^) + Y}(ffifcn4)( ^L^Xi -r k ) (/lifo) - (x t | ^)), (5.2) 



if 

inimize ^ Afc) + 
together with the dual problem 



K \ K 



minimize V (/? □ /£) - V (^(t; fc ) +^(« fc ) + (v k \r k )). (5.3) 

Remark 5.2 Problem |5 . 1 1 extends significantly the multivariate minimization framework of |j3 |[T2ll . 
There, (/ij)i^ m were the zero function, (£k)l^k^K were the function and (g k )x^k^K were 
differentiable everywhere with a Lipschitzian gradient. Finally, no dual problem was considered. 



Proposition 5.3 Consider the setting of Problem \5.1\ Suppose that (I5.2D has a solution, and set 



l^k^K 

Then (I5.1D is satisfied in each of the following cases. 



(Vi G {l,...,m}) Xi G dom/j 
(Vfc G {1, . . . ,#}) y fc G dom^ + dom4 



(5.4) 
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(i) (nOi^A- G sri£. 

(ii) I? — (rfc)i^fc^ic is a closed vector subspace. 

(iii) For every i E {1,... , m}, fa is real-valued and, for every k G {1, . . . , -K"}, the operator 0Jli %j — >■ 
£fc : (^)i<i<m ^ EjLi ^ surjective. 

(iv) For every G {1, . . . , -K"}, or 4 is real-valued. 

(v) ("%)i^ m and (£/fc)i<fccK are finite-dimensional, and (Vi G {1, . . . , m})(3xj G ridom/j)(VA; G 
{1, . . .,K}) YT=i L kiXi ~r k £ hdomg k + ridom4. 

Proof. Define ~H and Q as in (I2.5D . and L, z, and r as in (12.61) . Set 



J/:-H-»]-oo,+°o] = * ^ E£i and ^ R: a; ^ h M), 

\o- Q ->■ ]-oo,+oo] : y h-> Ylk=i9k(Vk) and € : £ ->• ]-oo, +oo] : y i-)- i k (y k ) 

Then (f574b and HI Proposition 12.6(ii)] yield 



(5.5) 



E = { La; — y | a; G dom / and y G dom g + dom £} 

= Zy (dom/) — (dome/ + dom .£) (5.6) 
= L(dom(/ + h- (• | z))) - dom (gDl). (5.7) 



[(!)) Since the functions (4)i<fc<K are strongly convex, so is Hence, dom£* = £/ 
Propositions 11.16 and 14.15] and therefore IIH Propositions 15.7(iv) and 24.27] imply that 
dgUdl = d{gU£) and gU£ G r (</). On the other hand, ([5771) yields G sri(L(dom(/ + /i - 
(• | z))) — dom (#□£)(• — r)). Thus, we derive from |(8l Theorem 16.37(i)] that 

df + L* o (dgnd£ k ) o (L ■ -r) + Vh - z = d(f + h - {■ | z)) + L* o d(g D£) o (L ■ —r) 

= d(f + h-{-\ z) + {gUi)o{L--r)). (5.8) 

Since (15.2ft has a solution and is equivalent to minimizing f + h — (■ \ z) + (g U£) o (L ■ —r) over ~H, 
Fermat's rule HI Theorem 16.2] implies that G ran<9(/ + h - (• | z) + (gO£) o (L • -r)). Hence 
(15.8ft yields z G ran(<9/ + L* o (<9g □ 54) o (L • -r) + Vh) and we conclude that (I5TTT) is satisfied. 

[ODHffll [8, Proposition 6.19(i)]. 

aii)H(i)} We have L(dom /) = L{U) = Q. Hence, (DOS) yields £ = 



(iv)|=>fC0]: We have dom g + dom£ = Q. Hence, (15.61) yields E = Q. 



[(v)H(i)) Since dimC? < +oo, sriFJ = hE. On the other hand, by (15T61) and [U Corollary 6.15], 
hE = ri (L(dom/) — domgf — dom£) = L(ridom/) — ridomg — ridom£. (5.9) 

Thus, r G sriFJ 44> (3x G ridom/ = X^^idomfa) Lx—r G ridomg+ridom^ = xfL^ridom + 
ridom4). □ 

Proposition 5.4 Consider the setting of Problem \5.1\ For every i G {l,...,m}, Zet (ai,i, n )neN> 

(bi,i,n)neNj ari( i (ci,i, n )neN be absolutely summable sequences in Tii and, for every k G {1, . . . ,K}, 
let (a2,fc,n)neN; (&2,fc,n)neN;> an d ( c 2,fc,n)neN be absolutely summable sequences in Q k . Furthermore, let 



19 



xi,o G Hi, x mfi G ?{ m , «i )0 eGi, v K ,o G £k, Zet e G ]0, l/(/3 + Zet (7 n )neN &e a sequence 
in [e, (1 — e)//3], and set 



For n = 0, 1, . . . 



For i = 1 , 


. . . ,m 








Tn^VM^n) + 2_^fc=l L ki v k,n + 0>l,i,n) 


Pl,i,n = 


P rOX 7„ 


fi( s l,i,n +1nZi) + 6l,i, n 


For k = 1 






S2,fc,n = 




7n,(V^(Ufc )Tl ) - L fe jX ijn + a 2 ,fe, n ) 


P2,fc,n = 


- s 2,k,n 


- 7n(^fc + prox 7 -i 9fc (7- 1 s 2 ,fc, n - J" fc ) + & 2 ,fc,n) 


Q2,k,n = 


P2,k,n 


- 7n(V^(p 2 ,fc,n) - XX 1 L kiPl,i,n + C 2 ,fc, n ) 


Vk,n+1 : 


= V k ,n ~ 


- S2,fc,n + <72,fc,n 


For i = 1, 


... ,m 






Pl,i,n ~ 


" 7n(V/ii(pi ) i )Tl ) + ]fJfcLi L* ki p 2> k,n + Cl,i, n ) 




- %i,n ~ 


^l,i,n i Ql,i,n- 



(5.10) 



Then the following hold. 

(i) (Vie{l, . . . ,m}) En G N lki,n-Pl,i,n|| 2 <+00, and (VTcG {1,... ,#}) EnGN ll u fc,n-P2,fc,n || 2 < +00. 

(ii) There exist a solution (xi, . . . , x^) to (I5.2D and a solution (W[, . . . , to (I5.3D such that the 
following hold. 

(a) (Vi G {1, . . . , m}) Sj )Tl x7 and - ]Cf=i ^fci^ifc G dfi(%i) + Vhi(zj). 

(b) (Vfc G {1, . . . , #}) v fe , n ^ and £™ i ifei^ " r k G + V^(^). 

(c) Suppose that, for some j G {1, . . . , m}, fj or hj is uniformly convex at xj. Then x^ n — > xj. 

(d) Suppose that, for some I G {1, . . . , K}, g* or £* is uniformly convex at v[. Then vi >n — >■ vj. 

Proof. Set 

(V*€{l,...,m}) Ai = dfi and Q = Vh, 
(VfcG {1,...,^}) B* = 00fc and D k = d£ k . 

It follows from [8l Proposition 17.10] that the operators (Cj)i<gj<g m are monotone, and from [8l 
Theorem 20.40] that the operators (Aj)i^ TO , (B k )i^ k ^ m , an d {D^i^k^K are maximally monotone. 
Moreover, for every & G {1, . . . , K}, we derive from [8l Corollary 13.33 and Theorem 18.15] that 
£* k is Frechet differentiable on Q k and V££ is i^-Lipschitzian, and from [8, Corollary 16.24 and 
Proposition 17.26(i)] that D k 1 = {dl k )~ l = dt k = {Vt k }. On the other hand, (15711) implies that 
(I1.2D possesses a solution, and (I1.12D implies that (I5.10D is a special case of (I2.4D . We also recall 
that the uniform convexity of a function <p G Fq('H) at x G domcfy? implies the uniform monotonicity 
of cV at x Il46l Section 3.4]. Altogether, the claims will follow at once from Theorem 12 .41 provided 
we show that, in the setting of (f5"TTT) and (15.11ft . (fL2l) becomes (15721) and (fL3l) becomes ([5731) . To 
this end, let us first observe that since, for every k G {1, . . . , if}, dom = [8, Proposition 24.27] 
yields 

(VfcG {!,...,#}) B fc n£) fc = a^D54 = %fcn4), (5.12) 
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while [8, Corollaries 16.24 and 16.38(iii)] yield 

(V* G {1, . . . , K}) B- 1 + D k l = dg% + {Vt k } = d(g* k + t k ). (5.13) 
Likewise, using [8, Theorem 15.3], we obtain 

(Vi G {1, . . . ,m}) (Ai+d)- 1 = {dfi+VK)- 1 = (difi + hi))' 1 = d(fi+hi)* = d(f*ah*). (5.14) 

Now let us define W and Q as in (I2.5D . i, z, and r as in (I2.6D . and /, fa, 3, and ^ as in (I5.5D . We 
derive from (15TTTT) . (15TT21) . Corollary 16.38(iii), Propositions 16.5(h), 16.8, and 17.26(i)], and 
Fermat's rule [8, Theorem 16.2] that, for every x = (xi)i<jj<j m G 71, 

x solves (O) (Vi G {1, . . . , m}) G 9/i(x f ) + ^ f % fc □ 4) ( £ L kjXj - r fc ] ] 

fe=i ^ S=i 
+ Vhi(xi) - Zi 

«0£ 0/(sb) + J L*(a(gD£)( J La;-r)) +V{h-(- \ z))(x) 
^Oed(f + (gD£)o(L- -r)) + h - (• | z)^ fas) 



44> a; solves (15721) . (5.15) 

Next, let t> = (vk)i^k^K G £/. Then we derive from (I5.13D . (15.14D , and the same subdifferential 
calculus rules as above that 

v solves (D & (Vk G {1, . . . , K}) G - L ki (d(f* □ h*) (z t - ^ L* H v^\ \ 

i=l ^ ^ i=l ' ' 

+ d{gl+t k + {-\r k )){v k ) 
&Oe-L(d(f*nh*)(z-L*v))+d(g* + £* + (■ \ r))(v) 

^Oed((f*nh*)o(z-L*-)+g* +£* + {■ I r))(v) 



^ v solves d573j), (5.16) 
which completes the proof. □ 



Remark 5.5 Proposition 15.41 provides a framework that captures and suggests extensions of multi- 
variate and/or infimal convolution variational formulations found in areas such as partial differential 
equations [4], machine learning [6], and image recovery |fT4l [TBI [38l ■ 



6 Univariate structured convex minimization problems 

Minimization problems involving a single primal variable can be obtained by setting m = 1 in Prob- 
lem 15.11 However, this approach imposes that infimal convolutions be performed exclusively with 
strongly convex functions. We use a different strategy relying on Proposition I3.2L which leads to a 
formulation allowing for infimal convolutions with general lower semicontinuous convex functions. 

Problem 6.1 Let % be a real Hilbert space, let K\, K 2 , and K be integers such that < K\ ^ 
K 2 ^ K ^ 1, let z G H, let / e T (W), and let h : H ->• R be convex and differentiable and such 
that V/i is jU-Lipschitzian for some /i G [0, +00 [. For every integer k G {1, . . . , K}, let ^ be a real 
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Hilbert space, let r k G Q k , let g k G T (g k ), let tp k G r (£ fe ), and let L fc G ¥>(?{, G k ); moreover, 
if ifi + 1 < k ^ i^2; <^fc is differentiable on Gk and such that V(p k is /3fc-Lipschitzian for some 
/3 k G [0, +oo[, and, if K2 + 1 ^ k ^ K , ip k is l//3 k -strongly convex for some (3 k G ]0,+oo[. Set 

j3 = max{/i, . . . , Pk) + + Y%=\ \\ L k\\ 2 , and assume that 

K 

z G ran (df + Y^ L t° ( d 9k Odcp k )o (L k ■ -r k ) + V/i) (6.1) 
fc=i 

and 



(Vfe G {l,...,if 2 }) G sri (dom g k — dom <p k ). (6.2) 
Solve the primal problem 

K 

minimize f(x) + (g k □ ip k ){L k x - r k ) + /i(x) - (x | z), (6.3) 
xew f — * 

k=l 

together with the dual problem 

(K \ m 

z-Y] L* k v k ) + V (g* k (v k ) + ^(vfc) + (v k I r fe » . (6.4) 

Remark 6.2 It follows from (Ol) and (H Propositions 11.16, 14.15, 15.7(i), and 24.27] that 

(Vfc G {1, . . . , K}) g k Dip k £T (g k ) and dg k Udip k = d{g k Uip k ). (6.5) 

Hence, using the same type of arguments as in the proof of Proposition 15.31 we can deduce similar 
conditions for (16.11) to hold, e.g., that (16.31) have a solution and that (r k )i^k^K he in the strong 
relative interior of {(L k x — y k )i^ k ^K \ x £ dom/ and (Vfc G {1, . . . G dom^ + dom^j.}. 

Proposition 6.3 Consider the setting of Problem \6.1\ Let (ax,i,n)nen, (&i,i,n)neN» an< ^ (cx,x,n)nen be 
absolutely summable sequences in %. For every integer k G {1, . . . , K}, let (ci2,fc,n)neN> (&2,fc,«)«eNj> and 
(c2,fc,n)neN be absolutely summable sequences in Q k ; moreover, if 1 < k ^ Ki, Zet (&i,fc+i )n )neN &e an 
absolutely summable sequence in Q k , and, if K\ + 1 < k ^ K2, let (ai,fe+i, n )neN (ci,fe+i,n)neN be 
absolutely summable sequences in Q k . Let x G yi,o G Q\, yK 2 ,o e Qk 2 > v i,o 6 £l> 
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vk,o G 9k, let e G ]0, + 1)[, let (7„) n eN be a sequence in [e, (1 - and set 

For n = 0, 1, . . . 

Sl,l,n = x n - 7 n (V/l(x n ) + £)fc=l L* k v k , n + ai^ n ) 
Pl,l,n = prox 7n/ (si,i i „ + 7„z) + 6i,i, n 
./far *! = !,...,#! 

51, fc+l, n = ?/fe,n + lnVk,n 

Pl,k+l,n = P rox 7nIi 3 fc s l,fc+l,n + &l,fc+l,n 

52, fc,n = ^fc,n — ln(Vk,n ~ L k X n + a 2j fc ;n ) 

P2,k,n = S 2 ,k,n ~ ln{r k + pTOX^i^ S 2 ,k,n ~ T k ) + &2,fc,n) 

92,fc,n = £>2,fc,n ~ 7n(Pl,fc+l,n _ LkPl,l,n + C 2) fc ;n ) 
L ^fc )n+ i = Ufc >n — S2,fc,n + ?2,fc,n 

7/K 1 ^K 2 ,/orfc = i!: 1 + l,...,K 2 

51, fc+l, n = Uk,n - ln{^fk{yk,n) ~ Vk,n + 
Pl,fc+l,n = Sl.fc+l.n 

52, fc,n = ^fc,n — ln(Vk,n ~ L k X n + a 2;kl n) 

P2,k,n = S 2 ,k,n ~ ln{r k + prOX^i^ (7^ 1 S 2 ,fc,n ~ T k ) + 6 2 ,fc,n) ^ g-j 

Q2,k,n = P2,k,n ~ 7n[Pl,k+l,n ~ LkPl,l,n + C 2) fc ; „) 
L Vk,n+1 = Vk,n ~ S 2 ,fe,n + <72,fc,n 

IfK 2 ^K,fork = K 2 + l,...,K 

S 2 ,k,n = V ktn - Kn(^<Pk( V k,n) ~ L kX n + 02,k,n) 
P2,k,n = S2,fe,n - ln(r k + prox^i^ (j~ 1 S 2 ,k,n - r k ) + b 2 ,k,n) 
?2,fc,n = Pl,k,n ~ 7n (V Vfc(P2,fc,n) ~ L kPl,l,n + C 2) fc,n) 
L w fc,n+l = ^ifc.n — S2,ifc,n + ^.ifc.n 

= Pl,l,n - 7n(V/l(pi 5 i jn ) + X]f=l L kP2,k,n + Cl,l, n ) 
= — s l,l,n + Ql,l,n 

IfK 1 ^0,fork = l,...,K 1 

<ll,k+l,n = Pl,fc+l,n + JnP2,k,n 
I Vk,n+1 = 2/fc,n — Sl,fc+l,n + <7l,fc+l,n 

If Ki ^ K 2 , for k = Ki + 1, . . . , K 2 

Ql,k+l,n = ~~ 7n(^ ( /'fe(Pl,jfc+l,Ti) _ P2,k,n + Cl,fc+l,n) 

L Vk,n+1 = 2/fe,n ~~ s l,fc+l,n + Ql,k+l,n- 

Then the following hold for some solution x to (16.31) and some solution (uf, . . . , to (I6.4D . 

(i) x n x and (V& G {1, . . . «&. 

(ii) Suppose that f or h is uniformly convex at x. Then 

(iii) Suppose that, for some I e {1, . . . , K}, gj is uniformly convex at v[. Then vi >n — > W- 

(iv) Suppose that K 2 / K and that, for some I e {K 2 + 1, . . . , K}, (p* is uniformly convex at wj. Then 

Vl, n -> W- 

Proof. Using (16.5ft and the same arguments as in the proof of Proposition 15.41 we first identify 
Problem l6.1l as a special case of Problem l3.1l with A = df, C = Vh, and (V/c G {1, . . . , K}) B k = dg k 
and S k = df k . Using (I1.12D . we then deduce the results from Proposition 13 .21 □ 

We conclude this section with an application to the approximation of inconsistent convex feasi- 
bility problems where, for the sake of brevity, we discuss only the primal problem. 
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Example 6.4 In Problem IBTTl set K x = K 2 = K, z = 0, h = 0, / = 0, and, for every k G {1, . . . , K} 
rk = and = ic k , where Ck is a nonempty closed convex subset of Qj. with projection operator 
Pfc. In addition, suppose that 



(Vfc G {1, . . . ,K}) Aigmimpk = {0} , <pk(0) = 0, and G sri (dom tj^ — dom^) 



(6.7) 



It follows from [51 Proposition 15.7(i)] that the inflmal convolutions (tc k □ fk)isik^K are exact. 
Hence, (16.31) becomes 



minimize min tpAL^x — y^), 
sew f-fyfcSCfe 



(6.8) 



and it is assumed to have at least one solution. We can interpret (16.8ft as a relaxation of the (possibly 
inconsistent) convex feasibility problem 



find x G H such that (VA; G {1, . . . , K}) L k x G C fc . 



(6.9) 



Indeed, it follows from (16.7ft that, if ( 16.91 ) is consistent, then its solutions coincide with those of 
(16.81) . Furthermore, in view of (11.121) . Algorithm (16.61) can be written as 

For n = 0, 1, . . . 

Pl,l,n = X n - 7„ ( Xlfcll L *k V k,n + a l,l,n) 

Fork = l,...,K 

s l,k+l,n = Uk,n + lnVk,n 
Pl,k+l,n = P rox 7n¥ , fc Sl,fc+l,n + &l,fc+l,n 
S2,fc,n = Vk t n ~ ln\Vk,n ~ LkX n + a2,fc, n ) 

P2,fc,n = •S2,fc,n ~ 7n (-Pfe (7n 1 s 2,fc,n) + &2,fc,n) jq-j 
92,fc,n = £>2,fc,n — Jn{Pl,k+l,n ~ £fcPl,l,n + c 2,fc,n) 
. ^fc.n+l = ^fe,n — •S2,fc,n + <?2,fc,n 
31,1,71 = Pl,l,n - 7n ( £fc=l L kP2,k,n + Cl,l,n) 

= x n - pi,i )n + gi,i jn 
Forfc = 1,...,K 

9l,fc+l,n = Pl,fc+l,n + lnP2,k,n 
Vk,n+1 = Dk,n ~ Sl,fc+l,Ti + 9l,fe+l,n- 

By Proposition I6.3|l(i)l (^n)neN converges weakly to a solution to (16.81) if inf ng N7„ > and 
sup neN 7 n < (1 + J2k=i \\ L k\\ 2 y 1/2 - Now suppose that, for every k G {1, . . . ,iTj- 3 ^ =U,L k = \&, 
if fc = 1, and = cj£.|| • || 2 , where G ]0, +00 [, if / 1. Then J6.9D reduces to the feasi- 



bility problem of finding x G f] k =i and (16.81) reduces to the constrained least-squares relaxation 
studied in 111811 . namely, minimize Ylk=2 0J kdc ( x )- 
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